FROM JEFF JODOIN THE NEW SYSOP OF AWAUG BBS, 24 HRS.  202-561- 2475 comes the following files I requested for those of you interested in these areas.
 
 
The following is an edited transcript of the special GEnie IBM RoundTable 
Real Time Conference held June 15, 1986 to discuss micro communication 
protocols, especially Xmodem, Ymodem, and Zmodem.  Chuck Forsberg, 
 communications consultant, author of Professional YAM, and originator 
of the Ymodem and Zmodem protocols was the guest of honor.
 
-----
 
Welcome to the IBM PC Roundtable. 
Your RoundTable Leaders are:
    Paul Homchick (GE Mail: HOMCHICK)
    David Kozinn (GE Mail: D.KOZINN) 
and Charles Strom (GE Mail: STROM)
 
************************************** 
** BIG DEAL RTC devoted to ** 
** Communications Protocols with ** 
** special guest Chuck Forsberg ** 
** author of PRO-YAM, originator ** 
** of YMODEM and ZMODEM protocols ** 
** >TONITE< at 8:30 P.M. EDT ** 
**************************************
 
<[chas] STROM> Many moons ago (actually only a few, perhaps 1978 or 
1979), Ward Christensen decided that there was a need to develop a file 
transfer protocol usable for micro to micro communications. He did 
this in a vacuum, not knowing the way it was done on mainframes, and 
xmodem (really correctly called Christensen protocol) was born.  There 
have been a succession of "improvements" and frills culminating in some 
real added functionality known as YMODEM and now ZMODEM.  Take it away 
Paul
 
<[Paul] HOMCHICK> Xmodem is a receiver-driven protocol, by that I mean 
that the receiver controls the handshaking, it works like this.  When 
you want to start a transfer, you must start programs up on each end, 
it is not automatic like kermit and some other mainframe protocols. 
The sending end simply waits for the receiver to send it something it 
recognizes.
 
The original XMODEM was very simple, it had a vocabulary of two: an ACK 
(acknowledge) and a NAK (neg ack) So it started out by sending a series 
of NAKs to the sender, meaning I haven't gotten anything yet, go ahead. 
When the receiver saw a NAK, it sent its first block, which in XMODEM 
was 128 bytes surrounded by some control bytes.  128 seemed like a good 
number because it was the same as the sector size in CP/M -- the only 
micro operating system around at that time -- specifically, it sent a 
SOH (start of header) the block number (example: 1) the block number 
compliment in 8 bit arithmetic (255) the 128 bytes of data, and an 
8-bit checksum of the 128 byte data stream.  The checksum was one of 
the failings of early xmodem because is was a simple modulo 255 sum of 
all the bytes.  It was easy to have two different data streams come up 
with the same checksum, so errors could creep in.
 
On the receiving end, it looks for the SOH, the sector number and the 
compliment, does its own checksum on the data and compares it to the 
checksum byte it got.  If all was OK, it sends an ACK, if not: a NAK, 
and the sender is supposed to re-send.  Now (big breath...) When we are 
all finished, the sender transmits an EOT (end of transmission), and 
that would tell the receiver that was all there was.  The problems with 
all of this were....

  1) the checksum was not too accurate, and

  2) noise on the line could send a false EOT to the receiver and
     prematurely cancel the transfer... usually about 250K into a
     300K xfer (heh heh).
 
It's interesting to note that only just now are commercial programs 
beginning to offer CRC instead of checksum, which brings up the next 
XMODEM improvement.
 
Any questions so far?
 
<[Mike] SCHEINBERG> If checksum was a failure, then why is checksum 
used alongside CRC for XMODEM?  Also, what is the difference between 
checksum and CRC?
 
<[Paul] HOMCHICK> Checksum is used because it was the only method 
officially documented by Ward Christensen for a long time, even though 
the PD programs were using CRC....
 
<[Mike] SCHEINBERG> So CRC is still not official?
 
<[Paul] HOMCHICK> Ward has said that he wishes he had taken a few days 
longer to develope the protocol and not used the simple checksum, as 
the differences.  CRC *IS official.  In the YMODEM doc, which has 
Ward's latest thinking on the subject, he endorses CRC.
 
The CRC is a polynomial.  Can chas or david help with what it is 
exactly?  If not, I'll try from memory.
 
<[David] D.KOZINN> It's a CCITT Polynomial, that's about all I 
remember.
 
<[Peter] P.HANDSMAN> One major improvement in ANY protocol would be a 
'start from any block' I don't know just how many times I lost a file 
after 400 blocks, is this being considered?
 
<[Paul] HOMCHICK> Zmodem has that capability. but let me talk about the 
CRC real quick.
 
<[Paul] HOMCHICK> Chuck, we were just discussing the history of micro 
protocols, had got to the introduction of CRC, care to carry on with 
CRC and Ymodem, then Modem?
 
<[Chuck] CAF> well - what is there to say about CRC -
 
<[Paul] HOMCHICK> I mentioned the problems with the single EOT and the 
error in Checksum.
 
<[Chuck] CAF> Most of the errors in XMODEM don't involve the CRC, but 
the weak spots in the protocol that have been discussed previously. 
YMODEM isn't any better, it's faster in networked situations because of 
the longer block length. Of course, on Genie having XMODEM in the nodes 
helps quite a bit.  Then, earlier this year, one of the VAN packet 
switched networks commissioned me to develop ZMODEM.
 
<[Paul] HOMCHICK> Ulp, too fast!  Please explain Ymodem.
 
<[Chuck] CAF> well, there are several "levels" of YMODEM, YMODEM is a 
set of extensions to XMODEM.  First historically was the batch 
transmission, with the file name sent in a block number 0.  Next, when 
I was downloading files from Unix systems, came the 1k blocks.  The 1k 
blocks are signified by starting each block with 002 instead of 001 
(SOH).  Full YMODEM also sends and uses the file length and 
modification date in the header packet.  This allows the receiver to 
throw away the extra garbage that XMODEM based protocols otherwise add 
to files at the end.  Questions??
 
<[Van] AVFLOWERS> Ok if you increased the size of the Xmodem blocks 
from 128 to a bigger number would the transfer be any faster?, if so 
why don't we do that.
 
<[Chuck] CAF> With Xmodem in the nodes, the speed difference isn't so 
great.  Perhaps it would be better to go all the way to ZMODEM, which 
gives other advantages.  Next ??
 
<[don] D.GOLDMAN> On Crosstalk, I have the option of selecting the 
block size.  What should I use?
 
<[Chuck] CAF> Crosstalk's proprietary protocol is incompatible with 
XMODEM or anything that is in the public domain.  Their XMODEM protocol 
is rather primitive.  It doesn't even support CRC.  next??
 
<[charles] C.DENGLERE> my question is there not any type of automatic 
file transfer protocol that will set the block size to the most 
effective transfer size and use that as a means of sending the file?
 
<[Chuck] CAF> The most effective block size is not always something 
that the programs can divine.  More comments when we get to ZMODEM. 
next??
 
<[KEN] KWSCHILDT> I am trying to use a peek to read a memory location 
to verify the carrier detect on a serial port?  any suggestions?
 
<[Chuck] CAF> the carrier status is not stored in any memory location. 
You need to do an input instruction to sample the carrier detect bit in 
the 8250 chip.  There may be other methods applicable to Basic I don't 
know of.  I don't know the syntax, but do recall there was some sort of 
in and out instructions in basica.  Would have to do some research and 
experimentation to find out more.  next??
 
<[David] D.KOZINN> Ok, Chuck, please tell us about Modem.
 
<[Chuck] CAF> Early this year one of the packet switched networks 
commissioned me to develop a successor protocol to XMODEM/YMODEM that 
was suitable for operation over packet switched networks.  They have 
been unhappy with the low throughput users were getting with XMODEM, 
and the 1k blocks of YMODEM were not a total solution.  Kermit wasn't 
the greatest (Kermit Sliding Windows) because of high overhead and 
inability of using sliding windows on some types of operating systems. 
The first thinking was to make more changes to XMODEM, but this seemed 
less and less desirable as the planning went on.  You've heard the old 
saw about the pig's ear and silk purse ... So, I reset to zero, so to 
speak. I wanted to refrain from new technology, so used basic routines 
that would be available to one used to writing XMODEM/CRC programs.
 
The structure of blocks is completely different, EVERYTHING is 
protected by CRC-16.  Well, almost everything, five Ctrl-X in a row is 
recognized as an abort so one can keyboard a cancel sequence.  I don't 
want to get in to too much detail here, but there are two types of 
header packets, binary and hex.  The receiver only sends the hex type, 
since flow control (X-on, X-off) may be in effect.  The sender can send 
either.  A binary data packet - one or more - can follow certain binary 
header packets.  Here is where things get interesting.  The sender 
sends an initial hex packet that wakes up the receiver, if the receiver 
isn't already running.  The receiver then sends a receive-init packet, 
which tells the sender what the receiver can handle (more later).  The 
sender then (in simplest form) sends a FILE header packet, followed 
with a binary data packet containing the file name, length, 
modification date, etc.  The length is used for display only, the 
actual file length is that which is sent. The receiver responds to 
this, then the sender sends a DATA header and data packets. There are 
four things that may happen at the end of each data packet (assuming no 
errors):
 
1.  The receiver sends an ACK and waits for the next header
 
2.  The receiver sends an ACK, and accepts more data packets without
    pause
 
3.  The receiver sends nothing, and accepts further data packets
 
4.  The receiver sends nothing, waits for another header.
 
Here's where the "magic" starts..........
 
If the receiver told the sender it can write to disk and do serial i/o 
at the same time. the sender keeps pumping data packets without waiting 
for any ACK.  In event of error, the receiver interrupts the sender 
with a header containing the byte count it has received OK. The sender 
seeks to that location in the file, and resumes. If the receiver 
indicated inability to do serial i/o and disk i/o, the sender tells the 
receiver to ACK each N bytes, where N was sent by the receiver to the 
sender in the receive init header.  This mode of operation is similar 
to YMODEM. Since a number of data packets may form a frame (beginning 
with the DATA header) there are some possibilities for tradeoffs.
 
<[chas] STROM> how can N range?
 
<[Chuck] CAF> N is sent to the sender , 0 implies infinite buffer, 
anything up to 32k gives the length before one must force an ACK 
response.
 
Back to packets... When using 1200 baud, the default is 256 byte 
packets, so an error is detected usually within 256 bytes, and 
transmission resumes.  When the sender gets to the end of the file, it 
sends a ZEOF header with the ending byte position.  If the receiver 
agrees, it closes the file, otherwise tells the sender to go back to 
the last good byte.  Any questions so far??
 
<[Paul] HOMCHICK> how hard is this to implement, could it be popular 
with modem program writers?
 
<[Chuck] CAF> The goal was to make ZMODEM easy to implement, since that 
was the main reason for XMODEM being so popular.  I'm not as happy with 
the complexity as one could wish, there seems to be a minimum level of 
complexity of protocol that will give RELIABLE transfers and also 
accommodate the various environments.  I could counter that XMODEM is 
very difficult to implement a robust accurate version, there are a 
number of secret tricks in Pro-YAM X/YMODEM to improve the robustness 
(ability to finish a transfer) and integrity/accuracy.  Most programs 
are not too forgiving of poor phone lines.  Providing C code should be 
a help, and I don't use any new CRC which would cause a problem.
 
<[don] D.GOLDMAN> is error checking the same with xmodem as modem?
 
<[Chuck] CAF> ZMODEM uses the same CRC-16 that XMODEM CRC uses.  It is 
used to protect everything, not just the data, so ZMODEM is more robust 
and accurate than XMODEM without requiring Pro-YAM's tricks. next?
 
<[don] D.GOLDMAN> since the block size id larger, how is error checking 
affected
 
<[Chuck] CAF> The CRC-16 polynomial used is considered reliable for 
block lengths up to about 4k.  The longest packet length in ZMODEM is 
1k, and may be shortened to 64 without too much loss in efficiency. 
The tradeoff here is speed of recovery from line hits versus overhead 
of send CRC more often.
 
<[Paul] HOMCHICK> still, since there is no ack/nak turnaround delay, 
the efficiency can be kept high.
 
<[Mike] SCHEINBERG> Would ZMODEM be able to have one computer be called 
and then at one appropriate time be able to send DATA without any 
problems? (Because It automatically sends packets, and automatic 
transfer starts.)
 
<[Chuck] CAF> On automatic calling, ZMODEM is very well suited for 
that, but that is also a function of the comm program that supports the 
protocol also. next??
 
<[chas] STROM> is the modem protocol description in the public domain? 
I know I have capability in Pro-Yam...  but where can I call (Modem at 
other end)?
 
<[Chuck] CAF> ZMODEM protocol is totally PD.
 
<[Chuck] CAF> Telegodzilla runs ZMODEM, [and soon we may see] ... 
PIBTERM and Red Ryder.  Also, the YAMDEMO.EXE program can be called 
with the DOS EXEC function from a BBS program for implementing ZMODEM 
transfer.
 
<[chas] STROM> (telegodzilla is Chuck's dial-in system)
 
<[Paul] HOMCHICK> Red Ryder?  have you been talking to Scott Watson?
 
<[Chuck] CAF> Yes, Scott gave me a call the other day.
 
<[Paul] HOMCHICK> (Red Ryder is the best MacIntosh comm program)
 
<[Chuck] CAF> Note I said possibly.
 
<[Paul] HOMCHICK> how about the network that commissioned it?
 
<[Chuck] CAF> ZMODEM is an application-to-application program, the 
network is transparent to it.
 
<[tom] TDAOUST> Robust Schmobust. A program is more efficient, faster, 
more accurate, more reliable but robust?  I had a woman that was 
robust, but she was none of the above.  Hmmm
 
<[Chuck] CAF> Ever had a XMODEM transfer quit because of line hits? 
That's what robustness (or the lack of it) is all about.
 
<[Al M] SERVU> is ZMODEM cp/m, dos or both
 
<[Chuck] CAF> It's mainly DOS and Unix, but could be implemented on 
CP/M. However, since most -nearly all- CP/M systems can't do disk 
writes while receiving data from the phone line, it can't do streaming.
 
<[Al M] SERVU> it can be used by XENEX V then?
 
<[Chuck] CAF> ZMODEM is actually four protocols in one, not counting 
fallback to YMODEM.  The distributed rz and sz programs are for Unix 
SYS III/V and 4.x BSD.
 
<[Al M] SERVU> is it available and if so where?
 
<[Chuck] CAF> The current version of these programs are in rzsz[12].shq 
on TeleGodzilla, and have been out on Usenet.  Guess I need to upload 
them.
 
<[Paul] HOMCHICK> that was my question, we will get rz and sz and put 
them here if OK with CAF.
 
<[Paul] HOMCHICK> Are they good on VMS, too?
 
<[Chuck] CAF> As for VMS, I can't get any doco on the QIO calls needed 
to do the sampling.
 
I might mention, The most popular aspects of ZMODEM are, 1) Only one 
command needed to transfer file(s) or download commands 2) ability to 
recover if a file transfer is interrupted, without having to start 
over, 3) insensitive to network delays, 4) robust transfers and rapid 
recovery from line hits. next ???
 
<JOHNCOOPER> CAF, I have all the docs DEC publishes on VMS's QIOs
 
<[Chuck] CAF> I could use info on QIO's for all the tty line magic. 
cd $Y head y1.c
 
<[David] D.KOZINN> Well, that about wraps up the formal part of the 
conference, I'd like to thank Chuck for taking the time to drop in.
 
<[Chuck] CAF> ok CUL.
 
** End ** 

